Operational Fidelity in Simulation-Based Training: The Use of Data from Threat and Error Management Analysis in Instructional Systems Design

A crucial aspect of the effectiveness of simulation-based training is the notion of fidelity. Typically, research and development in relation to simulation fidelity has focussed on achieving high levels of visual, kinaesthetic and functional realism. While this approach has lead to significant advancements in simulator-based training, there remains a need to develop mechanisms to ensure that training is responsive to the real operational needs of an organisation. This approach is termed “operational fidelity” and can be defined as ensuring the simulation is an authentic representation of the complex operational environment of an organisation. Methods currently used for understanding the operation of complex systems, such as modern aircraft, frequently adopt an abstracted approach to data acquisition. For instance, Cognitive Task Analysis is used to inform Instructional Systems Design through the decomposition and analysis of individual work elements. While these approaches to Instructional Systems Design provide a robust empirical method for the detailed analysis of system operation, they do not provide sufficient information to an organisation about the complex contextual factors that influence everyday operatoional performance. Accordingly, new tools need to be utilised in order to adopt a data-driven approach to simulator-based training. This paper describes in detail a new methodology to ensure enhanced simulator-based training through increased operational fidelity. Focussing on an example from the commercial aviation setting, the study details how data from the analysis of Threat and Error Management actions undertaken by flight crew during normal operations can be utilised in the development of scenario-based training. The paper outlines a four-stage process in scenario development and highlights the potential for such an approach to Instructional Systems Design in a variety of training environments.